feat(eventtemplates): add Preset event templates type and Quarkus-specific preset #733

andrewazores · 2024-12-04T16:05:34Z

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Read the contributing guidelines
Linked a relevant issue which this PR resolves
Linked any other relevant issues, PR's, or documentation, if any
Resolved all conflicts, if any
Rebased your branch PR on top of the latest upstream main branch
Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
Signed all commits using a GPG signature

To recreate commits with GPG signature git fetch upstream && git rebase --force --gpg-sign upstream/main

Depends on cryostatio/cryostat-core#486
Related to #548
See quarkusio/quarkus#44976 for disabled tests

Description of the change:

Adds a new PRESET event template type. These are intended to be readonly .jfc files that ship with Cryostat (included in the source repository or layered on top of the container at build time). This is somewhat similar to declarative (custom) event templates, but with the distinction that custom event templates can be created and deleted by the user at runtime. Preset templates are runtime immutable and are intended to do things like ship customized preset templates suggested by the Cryostat authors or community, or to enable support for popular application frameworks.

The first pass of this contained two different presets: Quarkus_Continuous and Quarkus_Profiling. These were based on the default continuous/profiling .jfc files shipped with OpenJDK 21, with the Quarkus-specific events enabled additionally. The latest revision of this PR includes a single tiny Quarkus preset template, which only enables the Quarkus-specific event types. In this latest revision the intent is that the user can use the preset to enable those particular events, and can achieve a layered effect by starting a second (or nth) recording using another target or custom event template. If the user wants to be able to do this with one recording in the future, then they can download the preset and use it as a basis to create their own custom template that enables all of the events in one go.

How to manually test:

Check out this PR, and check out feat(eventtemplates): support preset event template type cryostat-web#1497 in src/main/webui, check out related -core PR and ./mvnw clean install it
Build image
./smoketest.bash -O
Open UI
Select self-target, go to Events. Go to Event Templates and verify that a Quarkus template appears.
Go to Event Types and search for quarkus. Three event types should appear.
Create a custom recording on Cryostat with the Quarkus template preset
Create a custom recording on Cryostat with another (ex. Continuous) target event template
Wait some time for data to be collected. Click around the UI to generate more events.
Ensure that usual recording functions (archiving, View in Grafana, report generation) work on both of the recordings
Download both recordings
jfr print --events 'quarkus.*' quarkus.jfr and ensure that Quarkus events are present in the recording
jfr print --events 'quarkus.*' continuous.jfr and ensure that Quarkus events are present in this recording too
jfr summary quarkus.jfr and ensure that other JDK events are also present in the recording

quarkus.Rest {
  startTime = 16:35:01.786 (2024-12-04)
  duration = 0.0485 ms
  traceId = "566293e10cb594d71d65a2427ff35121"
  spanId = N/A
  httpMethod = "GET"
  uri = "/health/liveness"
  resourceClass = "io.cryostat.Health"
  resourceMethod = "liveness"
  client = "10.89.14.6:47586"
  eventThread = "executor-thread-2" (javaThreadId = 54)
}

quarkus.RestEnd {
  startTime = 16:35:01.786 (2024-12-04)
  traceId = "566293e10cb594d71d65a2427ff35121"
  spanId = N/A
  httpMethod = "GET"
  uri = "/health/liveness"
  resourceClass = "io.cryostat.Health"
  resourceMethod = "liveness"
  client = "10.89.14.6:47586"
  eventThread = "executor-thread-2" (javaThreadId = 54)
}

quarkus.Rest {
  startTime = 16:35:01.325 (2024-12-04)
  duration = 813 ms
  traceId = "fe68f4b3b3ff46078d5b535d44c261d8"
  spanId = N/A
  httpMethod = "GET"
  uri = "/api/v4/targets/2/reports/3"
  resourceClass = "io.cryostat.reports.Reports"
  resourceMethod = "getActive"
  client = "10.89.14.6:33376"
  eventThread = "executor-thread-5" (javaThreadId = 57)
}

quarkus.RestEnd {
  startTime = 16:35:02.138 (2024-12-04)
  traceId = "fe68f4b3b3ff46078d5b535d44c261d8"
  spanId = N/A
  httpMethod = "GET"
  uri = "/api/v4/targets/2/reports/3"
  resourceClass = "io.cryostat.reports.Reports"
  resourceMethod = "getActive"
  client = "10.89.14.6:33376"
  eventThread = "executor-thread-5" (javaThreadId = 57)
}

quarkus.RestStart {
  startTime = 16:35:08.857 (2024-12-04)
  traceId = "9b922ebf4434e3625013d0bb43072759"
  spanId = N/A
  httpMethod = "GET"
  uri = "/api/v4/activedownload/3"
  resourceClass = "io.cryostat.recordings.ActiveRecordingsDownload"
  resourceMethod = "handleActiveDownload"
  client = "10.89.14.6:33376"
  eventThread = "executor-thread-5" (javaThreadId = 57)
}

and

$ jfr summary quarkus.jfr 

 Version: 2.1
 Chunks: 2
 Start: 2024-12-06 17:13:10 (UTC)
 Duration: 15 s

 Event Type                                                             Count  Size (bytes) 
============================================================================================
 jdk.ModuleExport                                                        3837         39939
 jdk.BooleanFlag                                                         1010         30730
 jdk.NativeMethodSample                                                   766          9497
 jdk.ActiveSetting                                                        750         21728
 jdk.ObjectAllocationSample                                               497          7932
 jdk.JavaMonitorWait                                                      328          9650
 jdk.ModuleRequire                                                        326          3097
 jdk.LongFlag                                                             322         10440
 jdk.Checkpoint                                                           303       1288535
 jdk.UnsignedLongFlag                                                     264          8968
 jdk.OldObjectSample                                                      253          8270
 jdk.ThreadAllocationStatistics                                           242          2667
 jdk.ClassLoaderStatistics                                                240          5916
 jdk.MetaspaceChunkFreeListSummary                                        168          3272
 jdk.GCReferenceStatistics                                                168          1928
 jdk.PromoteObjectInNewPLAB                                               143          2510
 jdk.InitialEnvironmentVariable                                           132          7710
 jdk.NativeLibrary                                                        132          8837
 jdk.GCPhasePauseLevel1                                                   126          3728
 jdk.GCPhasePauseLevel2                                                   126          5266
 jdk.InitialSecurityProperty                                              118          8304
 jdk.GCHeapSummary                                                         84          3232
 jdk.MetaspaceSummary                                                      84          4408
 jdk.PSHeapSummary                                                         84         10153
 jdk.StringFlag                                                            56          1830
 jdk.ThreadPark                                                            51          1921
 jdk.InitialSystemProperty                                                 50          2678
 jdk.IntFlag                                                               50          1870
 jdk.GarbageCollection                                                     42           902
 jdk.YoungGarbageCollection                                                42           566
 jdk.GCCPUTime                                                             42           551
 jdk.GCPhasePause                                                          42           986
 jdk.ThreadCPULoad                                                         37           613
 jdk.DoubleFlag                                                            30          1210
 jdk.UnsignedIntFlag                                                       28           946
 quarkus.RestStart                                                         19          2136
 quarkus.RestEnd                                                           19           992
 quarkus.Rest                                                              19          1639
 jdk.ResidentSetSize                                                       16           278
 jdk.CPULoad                                                               16           310
 jdk.JavaThreadStatistics                                                  16           182

…cific presets

github-actions · 2024-12-04T16:05:50Z

This PR/issue depends on:

feat(templates): add preset event template type cryostat-core#486
By Dependent Issues (🤖). Happy coding!

andrewazores · 2024-12-06T19:47:28Z

New feature:

The target's JFR event types can be queried in match expressions. This should be particularly useful for Automated Rules combined with Preset Event Templates - Rules can be created which apply to targets based on the presence of registered JFR event types in the JVM, rather than only discovery-related attributes. The expression shown in this screenshot (eventTypeIds().exists(x, x.contains("quarkus")) can be used to test for Quarkus events present in the target JVM, and then we can infer that applying the Quarkus preset template to this target would make sense.

This requires the match expression evaluator to query the target for its event types, which may require a new connection to be opened to that target if none is already open in the connection manager cache, so this will slow down expression evaluations. However, the registered event types can be assumed to be static data, so the existing evaluator cache layer (<target, expression> -> bool) should prevent this from becoming too severe of a performance hit. The event type IDs are also only queried lazily if/when the eventTypeIds() function is called within the CEL script, so expressions that don't reference these will not cause a target connection to be opened and will not incur the performance penalty.

This would fit in nicely with #727 , as well as older ideas (can't find issue references now) to ship Cryostat with some pre-defined Automated Rules definitions. We could ship a Quarkus Preset Event Template, plus an Automated Rule that checks for quarkus.* event types as above and which enables the Quarkus Preset Event Template, so that the user can just toggle on that rule and start collecting data on any Quarkus applications that are discovered immediately.

andrewazores · 2024-12-06T20:01:01Z

With the cleanup in the latest commit, the required expression becomes slightly different: eventTypeIds(target).exists(x, x.contains("quarkus")). ie, the target instance must be passed to the eventTypeIds function.

andrewazores · 2024-12-06T20:02:14Z

I will split this out into a separate PR for ease of review and changeset management.

andrewazores · 2024-12-06T20:13:57Z

/build_test

github-actions · 2024-12-06T20:14:16Z

Workflow started at 12/6/2024, 3:14:16 PM. View Actions Run.

github-actions · 2024-12-06T20:16:04Z

CI build and push: At least one test failed ❌
https://github.com/cryostatio/cryostat/actions/runs/12205498322

andrewazores · 2024-12-06T20:17:50Z

Ah, CI failed because of the -core dependency not being merged yet.

feat(eventtemplates): add Preset event templates type and Quarkus-spe…

8fc7cd1

…cific presets

github-actions bot added the dependent label Dec 4, 2024

mergify bot added the safe-to-test label Dec 4, 2024

andrewazores mentioned this pull request Dec 4, 2024

feat(eventtemplates): support preset event template type cryostatio/cryostat-web#1497

Open

7 tasks

andrewazores added the feat New feature or request label Dec 4, 2024

enable quarkus-jfr

cbeba85

andrewazores requested a review from a team December 4, 2024 18:17

replace full presets with simple event enabling subset

5f50c5c

andrewazores changed the title ~~feat(eventtemplates): add Preset event templates type and Quarkus-specific presets~~ feat(eventtemplates): add Preset event templates type and Quarkus-specific preset Dec 6, 2024

andrewazores force-pushed the preset-templates branch from ed38823 to 5f50c5c Compare December 6, 2024 20:03

andrewazores mentioned this pull request Dec 6, 2024

feat(matchexpressions): expressions can reference target JFR event type IDs #735

Open

7 tasks

andrewazores marked this pull request as ready for review December 6, 2024 20:05

andrewazores requested a review from a team as a code owner December 6, 2024 20:05

andrewazores mentioned this pull request Dec 6, 2024

[Request] Support Hibernate ORM JFR events #736

Open

andrewazores added 7 commits December 6, 2024 16:14

test reliability

bfb9a55

fix event templates querying

e53df60

template type query tests

bda5fd5

schema

7f07cef

itest

2db5a69

disable broken itests

a9be613

remove log line

2180348

andrewazores mentioned this pull request Dec 6, 2024

[Bug] RulesPostJsonIT tests are disabled due to Quarkus bug #737

Open

andrewazores mentioned this pull request Dec 10, 2024

quarkus-jfr can cause a NullPointerException quarkusio/quarkus#44976

Open

andrewazores added docs needs-documentation labels Dec 10, 2024

andrewazores mentioned this pull request Dec 11, 2024

[Story] 4.0 updates cryostatio/cryostatio.github.io#202

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(eventtemplates): add Preset event templates type and Quarkus-specific preset #733

feat(eventtemplates): add Preset event templates type and Quarkus-specific preset #733

andrewazores commented Dec 4, 2024 •

edited

Loading

github-actions bot commented Dec 4, 2024

andrewazores commented Dec 6, 2024

andrewazores commented Dec 6, 2024 •

edited

Loading

andrewazores commented Dec 6, 2024

andrewazores commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

andrewazores commented Dec 6, 2024

feat(eventtemplates): add Preset event templates type and Quarkus-specific preset #733

Are you sure you want to change the base?

feat(eventtemplates): add Preset event templates type and Quarkus-specific preset #733

Conversation

andrewazores commented Dec 4, 2024 • edited Loading

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Description of the change:

How to manually test:

github-actions bot commented Dec 4, 2024

andrewazores commented Dec 6, 2024

andrewazores commented Dec 6, 2024 • edited Loading

andrewazores commented Dec 6, 2024

andrewazores commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

github-actions bot commented Dec 6, 2024

andrewazores commented Dec 6, 2024

andrewazores commented Dec 4, 2024 •

edited

Loading

andrewazores commented Dec 6, 2024 •

edited

Loading